Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New Rust re-write #27

Closed
wants to merge 36 commits into from
Closed

New Rust re-write #27

wants to merge 36 commits into from

Conversation

Stridsvagn69420
Copy link
Owner

@Stridsvagn69420 Stridsvagn69420 commented Oct 24, 2022

This will introduce the Rust re-write with v1.2.0, the managaement tools and obviously the hot new stuff as well as a re-write of the Wiki and better docs.

Checklist

  • Root crate documentation
  • Impl for Artist
  • Constructor for Artist
  • Impl for Album
  • Constructor for Album
  • Impl for Config
  • Wrappers/utils for Config
  • Impl for Hostinfo
  • Wrappers/utils for Hostinfo
  • Impl for Metadata
  • Constructor/utils for Metadata
  • Impl for Owner
  • Constructor for Owner

And then obviously the server stuff:

  • Meta headers (license, source code, Server Version)
  • Meta redirects (e.g. source code)
  • CORS everywhere
  • Hostinfo generator
  • Hostinfo and file serving routes with optional Basic Auth (directory listing will be simple <a href="filename.mp3">filename.mp3</a> stacks)

In the end, both cyrkensia and cyrkensia-cfg binaries should be finished in order to merge this PR!

  • cyrkensia - the Cyrkensia server
  • cyrkensia-cfg - tool for managing Cyrkensia-related files and folders

@Stridsvagn69420 Stridsvagn69420 added bug Something isn't working documentation Improvements or additions to documentation enhancement New feature or request help wanted Extra attention is needed dependencies Pull requests that update a dependency file labels Oct 24, 2022
@Stridsvagn69420 Stridsvagn69420 self-assigned this Oct 24, 2022
src/artist.rs Fixed Show fixed Hide fixed
@Stridsvagn69420
Copy link
Owner Author

Only the Server Routes and Root Docs needed now!

@Stridsvagn69420
Copy link
Owner Author

...for the raw library of course, not the binaries. But still, pretty neat!

@Stridsvagn69420
Copy link
Owner Author

Stridsvagn69420 commented Nov 12, 2022

Use actix-web shared state for the routes. Probably use a trait for it if possible.

Basically what's missing:

  • Shared state via Actix-Web (Hostinfo, Config, hostinfo timestamp, Htpasswd HashMap)
  • CORS everywhere (via middleware)
  • Password authentication (in file serving, via shared state)
  • The damn file serving routes
  • (NEW) Multiple root paths in config

@Stridsvagn69420
Copy link
Owner Author

The Password Authentication thingy will be done in the 3rd point on the checklist.

@Stridsvagn69420
Copy link
Owner Author

Only the file index and then actual file serving is missing. Thankfully these will be just two route functions, one for index, one for files.
Plus there are the index response (basically just formatting) and a 401 error response, similar to the sever_500() response.

Also, the file serving route should always include the size in the header, but also the SHA-256 and BLAKE3 hashes when there's a HEAD request (authentication has to be successful for this; if it isn't, always return the 401 response).

@Stridsvagn69420
Copy link
Owner Author

Using BLAKE3 is a good idea ...for checksums though.
I'll switch the password hashing to argon2 asap. BLAKE3 will still be used to create the file checksums, and SHA-256 will be removed.

@Stridsvagn69420
Copy link
Owner Author

Apart from bundling everything together as an executable and testing everything, only the file serving with the HEAD meta and optional HTTP Basic Auth is missing!

@Stridsvagn69420
Copy link
Owner Author

Stridsvagn69420 commented Dec 3, 2022

Aw shit, I made an "it's 2 AM and I'm sick and tired"-oopsies
image

  • It should read from the config's roots and check where that folder is.

@Stridsvagn69420
Copy link
Owner Author

It's only the actual fileserving-basicauth-headmeta route thingy now.

@Stridsvagn69420
Copy link
Owner Author

I should probably also remake the wiki before this gets merged...

@Stridsvagn69420
Copy link
Owner Author

Header stuff for the file serving:

  • ETag and Digest headers will contain BLAKE3 hash
  • Last-Modified header will use file metadata for time
  • Implement If-Modified-Since header behavior (client can send request with date to check if new data has to be downloaded)
    And HEAD request will only be exclusive to the fact that you can get the file size and the other metadata without actually dealing with megabytes or even gigabytes of wasted data. Plus, I want the HTTP codes to align with that's actually happening. So 200 for everything normal and 304 for when there's nothing new.

I have the feeling Actix-Web already does a lot of things for you, but I want to be sure and implement it my way, plus only something like the Actix-Web-Static-Fileserver thingy can really do things like hashing, ETag and Last-Modified, plus there's the user authentication (without auth, it won't send any information).
I originally also wanted to cache all of these things, but BLAKE3 is blazingly fast and the other caching mechanics get near the level of what NGINX and Apache can kindly do for you.

src/server/routes.rs Fixed Show fixed Hide fixed
src/server/routes.rs Fixed Show fixed Hide fixed
src/server/routes.rs Fixed Show fixed Hide fixed
let mut range = binding.split('-');

// Parse range
if let (Some(x), Some(y)) = (range.nth(0), range.nth(0)) {

Check failure

Code scanning / clippy

called `.nth(0)` on a `std::iter::Iterator`, when `.next()` is equivalent

called `.nth(0)` on a `std::iter::Iterator`, when `.next()` is equivalent
let mut range = binding.split('-');

// Parse range
if let (Some(x), Some(y)) = (range.nth(0), range.nth(0)) {

Check failure

Code scanning / clippy

called `.nth(0)` on a `std::iter::Iterator`, when `.next()` is equivalent

called `.nth(0)` on a `std::iter::Iterator`, when `.next()` is equivalent
@Stridsvagn69420
Copy link
Owner Author

Random stuff, but make the digest and partial content thing a middleware (has to be combined because digest depends on content).

@Stridsvagn69420
Copy link
Owner Author

Honestly, I might just make a new Rust rewrite.
I wrote it down on the Nucleus server, so time to post it here too.


was messing around with GTA's Self Radio, FFmpeg, audio codecs and tags in audii files, and I might make a rust-new-version of my WIP rust-version of Cyrkensia

you see, aside from having to redo most things because mixing basic file serving, sending metadata and password protecting everything together being complicated, I might do the storing part different

  1. I'll probably use SQL and will give the option to use SQLite or RDBMS. There will still be a config file in Json, but it's very simple.
    The database will have tables for:
    -maintainers and server admins
    -user database (Auth can be toggled in settings)
    -soundtrack table (lots of metadata and the filesystem info, maybe also hashes for integrity and stuff)
  2. I'll keep the current updating system for the soundtrack table btw, but the idea is that instead of having 300 MB of hostinfo.json, there will be a REST api endpoint for getting the file metadata and in general to know what the server even offers
    and 3. (I forgot this because slewpy)
    there should be a little tool that updates the files' tags baaed on the database

I have a more elaborate version on this now.
So, I'm gonna make it based on the current architecture, BUT the hostinfo won't be so fucking huge anymore. It merely lists the audio files, maybe even just the groups (what I called albums), and of course the server meta.
And then there's another endpoint, the songinfo, that allows the client to read more data about a file. And this part just reads the available data from the database (probably SQL then). The database can be manually filled, but if you already have metadata on your file, another tool tries to write it to the database on command. Idk if I will include caching, probably yes if I figure out how.

So things will get debloated and the architecture will get better too. I just have to find time when I can actually do it, school and stuff you know.

@Stridsvagn69420
Copy link
Owner Author

Auth and Content hashing meta thingy will most likely be a middleware btw.

@Stridsvagn69420
Copy link
Owner Author

Gonna rewrite the entire thing again...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working dependencies Pull requests that update a dependency file documentation Improvements or additions to documentation enhancement New feature or request help wanted Extra attention is needed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Core] Make the code base prettier or switch to new language entirely
1 participant